[EP API] header-only adapter for EP API by fs-eire · Pull Request #26919 · microsoft/onnxruntime

fs-eire · 2026-01-06T08:12:56Z

Description

This PR adds a few headers for supporting building WebGPU EP and CUDA EP as plugin EPs.

See summary of #26907

adrianlizarraga · 2026-02-04T00:43:47Z

@fs-eire, thank you for adding these utilities. We currently have some plugin EP utilities in this directory: https://github.com/microsoft/onnxruntime/tree/main/include/onnxruntime/core/providers/utils

And there's another on the way here: #25753

Could we consolidate all of these utilities into one location?

include/onnxruntime/ep/_pch.h

edgchen1 · 2026-02-04T01:46:18Z

include/onnxruntime/ep/README.md

+
+### Usage
+
+Make sure to include "ep/_pch.h" for all source code in the implementation. Using PCH is recommended.


Using PCH is recommended.

does this mean something other than including "_pch.h"? it's not obvious to me.

perhaps I'm just not understanding the name. could you either explain it more here or give _pch.h a more descriptive name?

_pch.h is the file, but we want to actually use compiler flags to specify the file as a PCH to enforce.

I probably should write Using PCH compiler flag is recommended.

updated comments to make it clear.

edgchen1 · 2026-02-04T01:58:35Z

include/onnxruntime/ep/adapter/allocator.h

+
+#pragma once
+
+#include "core/framework/allocator.h"


naive question - is this header intended to be used by a plugin EP which doesn't have access to internal ORT code? if so, how can we use internal ORT headers?

No. The whole purpose of the folder include/onnxruntime/ep/adapter is designed for a very specific goal: to support the existing EPs (majorly WebGPU EP and CUDA EP) to be able to use the EP API, with minimal code changes.

This means most of the existing code can be kept as-is. For some components, we need a "reverse bridge" to support an implementation of OrtAllocator which wraps an ORT internal allocator, like this.

edgchen1 · 2026-02-04T02:01:56Z

@fs-eire, thank you for adding these utilities. We currently have some plugin EP utilities in this directory: https://github.com/microsoft/onnxruntime/tree/main/include/onnxruntime/core/providers/utils

And there's another on the way here: #25753

Could we consolidate all of these utilities into one location?

it may be worth considering whether the utility is generally useful to any plugin EP vs. useful to plugin EPs within the ORT repo. some of these seem like the latter.

fs-eire · 2026-02-04T08:02:46Z

@fs-eire, thank you for adding these utilities. We currently have some plugin EP utilities in this directory: https://github.com/microsoft/onnxruntime/tree/main/include/onnxruntime/core/providers/utils
And there's another on the way here: #25753
Could we consolidate all of these utilities into one location?

it may be worth considering whether the utility is generally useful to any plugin EP vs. useful to plugin EPs within the ORT repo. some of these seem like the latter.

I agree. we should not have separated implementation in different places of the repo.

In general, I think there are 2 different types of headers:

the headers that ONLY depends on ORT public headers. This can include some useful common utils/macros and helper functions.
the headers also depending on ORT internal headers. Specifically, for the EP adapter implementation. (currently they are in include/onnxruntime/ep/adapter)

I would recommend to put them somewhere under include/onnxruntime/, because considering the usage header-only will be much more easier for users to use. And if it's header-only, it should be put inside include/onnxruntime/.

All other details are open to discussion.

adrianlizarraga · 2026-02-06T22:49:07Z

include/onnxruntime/ep/adapter/allocator.h

+    auto* allocator = static_cast<const Allocator*>(this_ptr);
+    return allocator->memory_info_;
+  }
+


Just want to double check that AllocOnStream is not also needed for cuda or webgpu EPs.

we can always extend those classes to support more function that may be missing for CUDA. The classes are generally designed to be generic enough for being reused in a future migration for CUDA EP.

adrianlizarraga · 2026-02-06T23:02:48Z

include/onnxruntime/ep/adapter/ep.h

+  inline const DataTransferManager& GetDataTransferManager() const noexcept {
+    return data_transfer_manager_;
+  }
+  [[nodiscard]] Status GetTempSpaceCPUAllocator(AllocatorPtr* output) const {


GetTempSpaceCPUAllocator and GetTempSpaceAllocator are internally implemented by onnxruntime::OpKernelContext. Would it be possible/appropriate to add public C APIs to OrtKernelContext for these functions instead of including them here?

I would prefer to keep it in the current way.

There are 2 reasons:

performance consideration. Here, onnxruntime::ep::adapter::EP::GetTempSpaceCPUAllocator accepts a AllocatorPtr*, which is a pointer to an internal type. We have access to the reference to the WebGpuExecutionProvider instance any way so it's not a problem of getting the actual allocator pointer returned by it.
If we change that to use an allocator returned by C API, then we have to use onnxruntime::ep::adapter::Allocator to wrap it (which has some overhead).

for usage inside include/onnxruntime/ep/adapter, there is no point to do this as long as WebGPU EP creates the allocator and use it both without the C API.

adrianlizarraga · 2026-02-06T23:10:43Z

include/onnxruntime/ep/adapter/kernel_registry.h

+      return ToOrtStatus(status);
+    }
+    *out = nullptr;
+    status = kernel->CreateControlFlowKernelImpl(info, out);


nit: maybe add a comment explaining why we try to create a control flow kernel first.

added, please review the comments.

include/onnxruntime/ep/adapter/op_kernel_info.h

include/onnxruntime/ep/api.h

…Impl

include/onnxruntime/ep/_pch.h

include/onnxruntime/ep/adapter/allocator.h

edgchen1 · 2026-02-12T01:59:05Z

include/onnxruntime/ep/README.md

+
+### Usage
+
+Make sure to include "ep/_pch.h" for all source code in the implementation. Using PCH compiler flag is recommended.


I still don't understand the name _pch.h. maybe you can explain it to me. or could we name it something like ep/adapters.h?

we are using EP_SPECIFIC_USING_DECLARATIONS to "override" some ORT classes with our adapter classes. This requires the header file to be included before any header that defines the old ORT classes. The best way to achieve this is to use the PCH (pre-compiled header), which is guaranteed by the compiler to ensure the file is included first.

we do have some 'pch.h' in winml folder, a 'test_pch.h' in cmake folder, and a PCH for CUDA EP: onnxruntime\core\providers\cuda\cuda_pch.h. Using the filename containing 'pch' may be a good way to indicating that the file is used as PCH explicitly.

The reason why the file is not inside adapter folder is because it depends on the shared plugin EP headers. Before we refactored them we probably need to keep it because otherwise #include "../common.h" seems not a very good way.

edgchen1 · 2026-02-12T02:00:05Z

include/onnxruntime/ep/README.md

@@ -0,0 +1,7 @@
+## EP adapter
+
+This folder contains a set of C++ header files. They are used specifically for allowing ONNX Runtime internal kernel-based EPs to use the plugin-style EP API while keep minimal changes to existing code.


this readme is not located in the adapters directory. maybe move it there, or explicitly name the adapters directory.

I added a section in the README to explain the folder structure. Since there is an ongoing discussion about unifying the shared headers, I expect the current folder structure to be refactored later.

in future, when we have a dedicated folder for the shared plugin EP headers, we can put everything in the adapter directory.

fs-eire force-pushed the fs-eire/ep-api-adatper-header-only branch from 74b2ed4 to b8a9132 Compare February 2, 2026 23:35

fs-eire mentioned this pull request Feb 2, 2026

[WIP] Make WebGPU EP compatible with EP API #26907

Draft

adrianlizarraga requested review from adrianlizarraga and edgchen1 February 4, 2026 00:44

edgchen1 reviewed Feb 4, 2026

View reviewed changes

adrianlizarraga mentioned this pull request Feb 6, 2026

[EP API] extract common code for EP API adapter #26879

Open

adrianlizarraga reviewed Feb 7, 2026

View reviewed changes

[EP API] header-only adapter for EP API

c9e4973

fs-eire force-pushed the fs-eire/ep-api-adatper-header-only branch from b8a9132 to c9e4973 Compare February 7, 2026 04:58

fs-eire added 4 commits February 6, 2026 21:13

resolve comments - std::unique_ptr -> std::optional

e6c2e8e

resolve comments - OrtKernelInfo -> OpKernelInfo in comments

4bfc650

resolve comments - add comment for why to use CreateControlFlowKernel…

c79d03d

…Impl

resolve comments - update README.md

e0e4037

edgchen1 reviewed Feb 12, 2026

View reviewed changes

fs-eire added 3 commits February 12, 2026 16:55

resolve comments - missing macro guard in allocator.h

f88218e

resolve comments - add the missing '#undef'

db67207

resolve comments - add info about folder structure in README.md

2247620


		### Usage

		Make sure to include "ep/_pch.h" for all source code in the implementation. Using PCH is recommended.

		@@ -0,0 +1,7 @@
		## EP adapter

		This folder contains a set of C++ header files. They are used specifically for allowing ONNX Runtime internal kernel-based EPs to use the plugin-style EP API while keep minimal changes to existing code.


		#pragma once

		#include "core/framework/allocator.h"

Conversation

fs-eire commented Jan 6, 2026

Description

Uh oh!

adrianlizarraga commented Feb 4, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

edgchen1 commented Feb 4, 2026

Uh oh!

fs-eire commented Feb 4, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants